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I.  INTRODUCTION 

Let  cP  = {F (x-u) ;ueA]  be  a location  parameter  family  of  distributions; 

where  A is  an  appropriate  subset  of  the  real  line  such  as  an  interval  or  a 

countable  collection  of  real  numbers.  A distribution  M(x)  of  a random 

variable  X is  called  a finite  location  mixture  on  if  and  only  if  there 

exists  a positive  integer  k,  positive  constants  p ,-p  , . . . ,p  with  sum 

1 d J£ 

equal  to  one;  and  distinct  distributions  FCx-u^FCx-Ug), . . . ;F(x-ufc) 
belonging  to  ; such  that 

k 

M(x)  = J p .F(x-u . ) . (l) 

i=l 

Suppose  that  U is  a discrete  random  variable  defined  on  A with  the 
probability  mass  function 

P (U=U^  ) = i=l;2;  . . ;k  . (2) 

If  A includes  zero;  then  the  distribution  F(s)  of  a random  variable; 
say  S;  generates  the  family  . Now  it  can  be  shown  easily;  by  using 
conditional  distribution  or  characteristic  function;  that  X is  distributed 
as  S+U  or  briefly  X = S+U;  where  S and  U are  independent.  We  assume 
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that  the  distribution  of  the  generator  random  variable  S is  completely 

known  and  the  distribution  of  U depends  on  the  unknown  parameters  p V s 

and  u. ' s. 
x 

Similarly  a distribution  N(y)  of  a random  variable  Y is  called  a 
finite  scale  mixture  if 


k 

N(y)  = ^PiG(y/vi)  . (3) 

i=l 

Here  the  distinct  distributions  G(y/v^), G(y/Vg ), . . . *G(y/v  ) belong  to  a 
scale  family  of  distributions  = (G(y/v): veB},  where  B is.  an  appro- 
priate subset  of  (0,°°)$;  p^’s  are  defined  as  before. 

Suppose  that  V is  a discrete  positive  random  variable  defined  on  B 
with  the  probability  mass  function 


P(V=vt)  = p±,  i=l,2,.,..,k  . 0) 

If  B includes  1,  then  the  distribution  G(t)  of  a random  variable, 

say  T,  generates  the  family  . It  is  easy  to  show  that  Y = TV,  where 

T and  V are  two  independent  random  variables, 
d X d S U 

From  X = S+U  we  have  e = e e , i.e. , a finite  location  mixture  can 
be  reduced  to  a finite  scale  mixture  by  the  exponential  transformation  y = e . 
Similarly,  from  Y = TV  we  have  log|Y|  = log|T|  + logV,  i.e.,  a finite 
scale  mixture  can  be  reduced  to  a finite  location  mixture  by  the  logarithmic 
transformation  x = logjy|. 

Representation  of  these  finite  mixtures  in  terms  of  random  variables  is 
quite  useful  for  generating  random  samples  and  for  analyzing  their  properties. 
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A finite  mixture  of  normal  distributions  with  the  same  known  variance 


is  an  example  of  a finite  location  mixture,  and  a finite  mixture  of  nega- 
tive exponential  distributions  is  an  example  of  a scale  mixture.  The  point 
estimation  of  the  parameters  in  a mixture  of  two  such  distributions  has 
received  some  attention  in  the  literature.  However,  when  we  have  a mixture 
of  more  than  two  of  these  distributions,  or  more  generally  a finite  location 
or  scale  mixture,  estimation  becomes  more  difficult. 

In  this  article  we  first  show,  by  a simple  argument,  that  finite  loca- 
tion and  finite  scale  mixtures  are  identifiable.  Next,  we  suggest  a method 
of  moments  for  estimating  the  unknown  parameters.  Finally,  as  an  example, 
we  apply  the  method  to  a finite  mixture  of  negative  exponential  distri- 
butions. 

2.  IDENTIFIABILITT  OF  FINITE  LOCATION 
AND  FINITE  SCALE  MIXTURES 

Definition.  A finite  location  mixture  given  by  (l)  is  identifiable  with 
respect  to  db  if  it  has  a unique  representation  as  far  as  the  mixed  distri- 
butions, their  number,  and  the  mixing  proportions  are  concerned.  In  other 
words , 

k k* 

M(x)  = £pF(x-u)=  £p'F(x-u')  (5) 

.1=1  0=1  3 3 

implies  k = k'  and  for  each  i there  is  some  j such  that  p^  = p^  and 

u.  = u'. . 
i 3 

In  terms  of  random  variables,  we  can  interpret  this  definition  in  the 
following  way:  if  X = S+U,  for  the  independent  random  variables  S and  U, 
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and  if  X = S+U* , for  the  independent  random  variables  S and  IP , 

(L 

then  the  distribution  of  X is  identifiable  with  respect  to  if  and 
only  if  U=U'. 

In  general,  estimation  problem  for  a mixture  makes  sense  if  the 
mixture  is  identifiable.,  We  can  easily  show  that  a finite  location 
mixture  is  identifiable  by  using  the  above  interpretation.  For  this 
purpose,  we  denote  the  characteristic  functions  of  S,  U,  and  Us  respec- 
tively by  cpg(t),  cpTJ(t, ) ^ and  cpy,  (t).  If  S+U  = S+U1,  then 

cps(t)cpu(t)  = cpg(t)cpXJ,  (t)  . (6) 

It  is  known  that  a characteristic  function  is  equal  to  1 at  t = 0,  and 
it  is  a continuous  function.  Thus  for  some  e > 0,  cpg(t)  ^ 0 and 

cPu(t)  = cput  (t)  (7) 

for  all  t in  the  interval  (-e,e).  Since  by  (2)  all  the  momehts  of  U 
exist.,  it  follows  from  (7 ) that 

E(un ) = E(U,n)  (8) 

for  all  non-negative  integers  n„  It  is  also  easy  to  show  that 


sup  < max  ( lu^  | , |u2 1 , . . . , luj  ) < 00  . 


n 


n 


(9) 


How.,  using  (8)  and  (9),  by  the  moment  problem  (see  [4]  page  182),  we  con- 


clude that  U = IP 
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The  identifiability  of  a finite  scale  mixture  follows  immediately 
from  the  identifiability  of  a finite  location  mixture  by  using  the  loga- 
rithmic transformation  we  referred  to  in  Section  1, 

The  identifiability  of  a finite  location  mixture  has  also  been  demon- 
strated by  Yakowitz  and  Spragins  [8]  by  using  a general  characterization 
theorem  regarding  the  identifiability  of  finite  mixtures. 

3.  ESTIMATION  OF  PARAMETERS 


To  estimate  the  parameters  of  a finite  location  or  scale  mixture  we 
apply  a moment  technique  similar  to  that  used  by  Bliscke[3],  and  Rennie  [5]. 
However , to  simplify  the  procedure,  we  often  use  the  relations  X = S+U 
and  Y = TV. 

Consider  a random  sample  X^,X^,...,X^  from  the  location  mixture  (l), 

and  denote  the  rth  moments  of  X,  S,  and  U respectively  by  a^,  b , and 

c with  d_  = b-  = c_  = 1.  We  assume  that  all  the  moments  in  question 

r 0 0 0 

exist.  Since  the  distribution  M(x)  depends  on  2k-l  unknown  parameters, 
we  use  the  first  2k-l  sample  moments  of  X,  i.e„. 


n 


a - ^ 
r 1 

i=l 


Exf/n 


r=l,2, . o . ,2k-l  , 


(10) 


to  estimate  the  unknown  parameters  . . . jU^PijPg* . . . ,p^. 

Taking  rth  moment  from  both  sides  of  X = S+U  and  using  the  inde- 
pendence of  S and  U,  we  have 


av»  = E ( -1  ) ^ r=0,l, . . . ,2k-l  . 


(11) 
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It  is  clear  that  the  matrix  B = (b.  .)  with 

iO 


b.  . = ( ^ ) b.  . , 

10  0 10 


i>  d— 0,1,  • • • > 2k-l  } 


= 0 


otherwise 


(12) 


is  a non-singular  matrix.  Now,  using  the  vectors  a = (aQ,a.,  . . . ’ 

c = (cq, e^,  . . . p)'^  an<l  ^e  ma'krix  ® defined  by  (12 ),  the  relations 

(ll)  can  be  written  as 


„ ,,-1 

a = Be  or  c = B a 


(13) 


Thus,  from  (13 ) and  (10 ),  we  can  find  the  moment  estimates  of  U in  terms 
of  the  moments  of  S,  which  are  known,  and  the  moment  estimates  of  X as 


r*t 

c = B a „ 


(14) 


It  follows  from  (2)  that  the  rth  moment  of  U is 


Using  (15),  with  r=0,l, . . . ,k-l,  we  obtain 


“1 

- 

1 

l ... 

1 

pl 

co 

u^ 

u2  ... 

u^ 

P2 

* 

0 

= 

C1 

k-i 

k-1 

k-1 

• 

_ul 

Up  • * • 

^ . 

_Pk_ 

_Ck-l  _ 

(15) 


(16) 


The  parameters  u-^u^, 
nomial 


can  be  assumed  to  be  the  zeros  of  the  poly- 
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p (x  ) = fl  (x-^)  = xk~  O' 1xk'1-  •••  - ak_1x-«k  . (17) 

1=1 

We  now  find  the  coefficients  0^,0^, ...  ,0!^  in  terms  of  c]_>  c2->  * * * >C2k-l’ 

It  is  obvious  that  i/p(u)  is  a zero  random  variable  for  any  non-negative 
integer  A,  Hence.,  we  have  E(l/p(U))  = 0 or 

“tci^-lC<€  * ■"  +“lC«.l  = Vk  for  . (18) 

From  (l8)  we  have  the  matrix  equation 


(19) 


The  matrix  in  (19),  by  using  (15),  can  be  expressed  as  WDW>  where  W = (w„), 

with  w. . = u1  for  i=0,l, . . . ,k-l,  j=l,2, . . . ,k,  is  the  Vandermonde  matrix 
1 J J 

and  D is  the  diagonal  matrix  with  P^Pg*  * * • on  main  diagonal. 

Since  p.  > 0 and  u.  / u.  for  i^  j,  we  conclude  that  all  these  matrices 

i l j 

are  non-singular.  Therefore,  by  solving  (19 ) we  can  find  the  coefficients 
of  the  polynomial  (15 ) and  then  the  roots  of  P(x)  = 0,  i.e.,  u^*s  in 

terms  of  c^s.  To  identify  these  roots,  we  can  assume,  without  loss  of 
generality,  that  < ■ • • < u^.  Substituting  the  moment  estimates  of 

c^s  in  (19),  we  find  uV  s,  i.e.,  the  moment  estimates  of  u^’s.  It 
should.be  noted  that  there  is  no  guarantee  that  we  have  distinct  real  uWs. 
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However,,  since  the  sample  moments  aJs  converge  to  a^'s  with  probability 
one.,  the  chance  of  having  imaginary  or  equal  u^'s  tends  to  zero  as  sample 
size  n tends  to  infinity. 

Replacing  the  moment  estimates  of  c^'s  and  tu’s  in  (l6),  we  obtain 
the  moment  estimates  of  p.’s. 

In  practice  we  may  not  know  the  distribution  of  Sj  however,  it  is  just 
enough  to  have  some  prior  knowledge  about  the  first  2k -1  moments  of  S, 

For -example,  suppose  that  a certain  material  is  produced  by  k machines 
whose  products  flow  automatically  into  a common  shipping  room.  Let  X be 
some  characteristic  of  the  material  and  assume  that  X has  the  same  distri- 
bution for  all  machines  when  they  are  new.  Thus,  we  can  say  the  distribution 
of  X and  S are  identical  at  the  beginning,  and  we  can  find  the  sample 
moments  of  S from  some  observed  values  of  X.  After  a period  the  machines 
become  old  and  there  would  be  some  different  shifts  in  X from  machine  to 
machine.  Now,  X for  the  products  in  the  shipping  room  has  a finite  loca- 
tion of  mixtures.  Observing  a random  sample  and  using  the  prior  knowledge 
about  S,  we  can  estimate  the  -unknown  parameters  by  the  above  procedure. 

To  estimate  the  parameters  of  a finite  scale  mixture,  we  first  find  the 
moment  estimates  of  V by  using  the  relation  Y = TV  and  then  we  apply  the 
above  procedure.  Actually,  since  T and  V are  independent  we  have 

E(Vr)  = E(Yr)/E(Tr)  . (20) 

« 

If  ECT1)  = 0 for  some  of  the  required  r’s,  we  cannot  use  (20), 

Here  we  transform  the  scale  mixture  to  a location  mixture  by  using  the  loga- 
rithmic transformation  we  referred  to  in  Section  1,  An  example  of  this  case 
is  the  normal  mixture 
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(21) 


M(x)  = ivM0,o-2) 
i=l 

in  which  T,  the  standard  normal  variable,  has  zero  odd.  moments, 

4,  A FINITE  MIXTURE  OF  NEGATIVE  EXPONENTIAL  DISTRIBUTIONS 

As  an  example  we  apply  the  above  method  to  find  the  moment  estimates 
of  a finite  mixture  of  negative  exponential  distributions  with  probability 
density  function 

k 

f(y)  = Epn.f,  (y)  , (22) 

i=l 

where 

ft(y)  = exp(-y/vi)/vi  . (23) 

This  distribution  is  often  used  for  the  analysis  of  the  completed 
length  of  service  [2],  and  it  has  also  some  application  in  reliability  and 
life  testing  [l].  For  the  case  k=2,  the  moment  estimation  of  the  param- 
eters has  already  received  some  attention,  for  example,  by  Rider  [5]  and 
Tallis  and  Light  [6].  But  for  k > 2 the  problem  becomes  more  involved 
and  there  is  no  practical  method  to  estimate  the  parameters. 

The  function  (22)  is  the  density  of  the  random  variable  Y = TV, 
where  T has  the  probability  density  function  exp(-t)  for  t >0  and  V 
has  the  probability  mass  function  given  by  (4).  We  know  that 

E(Tr)  = r(r+l)  = rl  (24) 

for  any  non-negative  integer  r.  Now,  consider  a random  sample 

Y ,Y  , ...,Y  from  a distribution  with  density  (22 ),  Taking 
-L  cL  n 
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(25) 


c = ECV21)  = E(Yr)/E(Tr)  = E(Yr)/r*  , 


Y for  X.,  and  v.  for  u.,  we  can  find  the  moment  estimates  by 
1 1*1  i 

using  (15) -(19). 

It  is  not  difficult  to  show  that  the  sampling  variability  of  the  sample 
moments  of  the  density  (23 ) becomes  larger  and  larger  as  we  use  higher 
moments.  This  behavior  of  sample  moments  reduces  the  efficiency  of  the 
estimates.  We  may  modify  the  method  by  taking  rth  fractional  moment  from 
both  sides  of  Y = TV.  To  estimate  the  parameters ^ for  v\  we  take 
w.  = and  for  r we  use  l/(2k-l),2/(2k-l), . . . , (2k-2)/(2k-l),l. 

Application  of  fractional  moments  may  be  also  useful  when  T does  not  have 
integral  moments. 


10 


References 


[1]  Barlow,  R.  E.  and  Proschan,  F.  (1975 )•  Statistical  Theory  of 
Reliability  and  Life  Testing.  Holt,  Rinehart  and  Winston,  Inc., 

New  York. 

[2]  Bartholomew,  D.  J.  (1973).  Stochastic  Models  for  Social  Sciences, 

2nd  ed.  John  Wiley  and  Sons,  New  York. 

[3]  Blischke,  W.  R.  (1962).  Estimating  the  parameters  of  mixtures  of 
binomial  distributions.  JASA,  59*  510-528. 

[4]  Breiman,  L.  (1968).  Probability.  Addis on -Wes ley  Publishing  Company, 
Reading,  Mass. 

[5]  Rennie,  R.  R.  (1974 )•  An  identification  algorithm  for  finite  mixtures 
of  distributions  with  common  central  moments.  Sankhya,  36,  315-320. 

[6]  Rider,  P.  L.  (1961).  The  method  of  moments  applied  to  a mixture  of 
two  exponential  distributions.  Ann.  Math.  St at.,  32,  143-147. 

[7]  Tallis,  G.  M.  and  Light,  R.  (1968).  The  use  of  fractional  moments 
for  estimating  the  parameters  of  a mixed  exponential  distributions. 
Technometrics,  10,  l6l-175. 

[8]  Yakowitz,  S.  and  Spragins,  J.  (1968),  On  the  identifiability  of 
finite  mixtures.  Ann.  Math.  Stat. , 40,  1728-1735. 


11 


TECHNICAL  REPORTS 


OFFICE  OF  NAVAL  RESEARCH  CONTRACT  N00014-67-A-0112-0030  (NR-0i*2~034) 


1.  "Confidence  Limits  for  the  Expected  Value  of  an  Arbitrary  Bounded  Random 
Variable  with  a Continuous  Distribution  Function,"  T.  W.  Anderson, 

October  1,  1969. 

2.  "Efficient  Estimation  of  Regression  Coefficients  in  Time  Series,"  T.  W. 
Anderson,  October  1,  1970. 

3.  "Determining  the  Appropriate  Sample  Size  for  Confidence  Limits  for  a 
Proportion,"  T.  W.  Anderson  and  H„  Burstein,  October  15,  1970. 

ii.  "Some  General  Results  on  Time-Ordered  Classification,"  D.  V.  Hinkley , 

July  30,  1971. 

5.  "Tests  for  Randomness  of  Directions  against  Equatorial  and  Bimodal 
Alternatives,"  T.  W.  Anderson  and  M.  A.  Stephens,  August  30,  1971* 

6.  "Estimation  of  Covariance  Matrices  with  Linear  Structure  and  Moving 
Average  Processes  of  Finite  Order,"  T.  W.  Anderson,  October  29,  1971. 

7-  "The  Stationarity  of  an  Estimated  Autoregressive  Process,"  T.  W. 

Anderson,  November  15,  1971- 

8.  "On  the  Inverse  of  Some  Covariance  Matrices  of  Toeplitz  Type,"  Raul 
Pedro  Mentz,  July  12,  1972. 

9.  "An  Asymptotic  Expansion  of  the  Distribution  of  "Student i zed"  Class- 
ification Statistics,"  T.  W.  Anderson,  September  10,  1972. 

10.  "Asymptotic  Evaluation  of  the  Probabilities  of  Misclassification  by 
Lineal'  Discriminant  Functions,"  T.  W.  Anderson,  September  28,  1972. 

11.  "Population  Mixing  Models  and  Clustering  Algorithms,"  Stanley  L. 

Sclove,  February  1,  1973. 

12.  "Asymptotic  Properties  and  Computation  of  Maximum  Likelihood  Estimates 
in  the  Mixed  Model  of  the  Analysis  of  Variance,"  John  James  Miller, 
November  21,  1973. 

13.  "Maximum  Likelihood  Estimation  in  the  Birth-and-Death  Process,"  Niels 
Keiding,  November  28,  1973. 

lU„  "Random  Orthogonal  Set  Functions  and  Stochastic  Models  for  the  Gravity 
Potential  of  the  Earth,"  Steffen  L.  Lauritzen,  December  27,  1973. 

15.  "Maximum  Likelihood  Estimation  of  Parameters  of  an  Autoregressive 
Process  with  Moving  Average  Residuals  and  Other  Covariance'  Matrices 
with  Linear  Structure , " T.  W,  Anderson,  December,  1973. 

16.  "Note  on  a Case-Study  in  Box-Jenkins  Seasonal  Forecasting  of  Time  Series, " 
Steffen  L.  Lauritzen,  April,  197^. 


TECHNICAL  REPORTS  (continued) 


17.  "General  Exponential  Models  for  Discrete  Observations," 

Steffen  L.  Lauritzen,  May,  1974. 

18.  "On  the  Interrelationships  among  Sufficiency,  Total  Sufficiency  and 
Some  Related  Concepts,"  Steffen  L.  Lauritzen,  June,  1974. 

19.  "Statistical  Inference  for  Multiply  Truncated  Power  Series  Distributions," 
T.  Cacoullos,  September  30,  1974. 


Office  of  Naval  Research  Contract  N00014-75-C-0442  (NR-042-034) 

20.  "Estimation  by  Maximum  Likelihood  in  Autoregressive  Moving  Average  Models 
in  the  Time  and  Frequency  Domains,"  T.  W.  Anderson,  June  1975. 

21.  "Asymptotic  Properties  of  Some  Estimators  in  Moving  Average  Models, 

Raul  Pedro  Mentz,  September  8,  1975. 

22.  "On  a Spectral  Estimate  Obtained  by  an  Autoregressive  Model  Fitting," 
Mituaki  Huzii,  February  1976. 

23.  "Estimating  Means  when  Some  Observations  are  Classified  by  Linear 
Discriminant  Function,"  Chien-Pai  Han,  April  1976. 

2k.  "Panels  and  Time  Series  Analysis:  Markov  Chains  and  Autoregressive 

Processes,"  T.  W.  Anderson,  July  1976. 

25.  "Repeated  Measurements  on  Autoregressive  Processes,"  T.  W.  Anderson, 

September  1976. 

26.  "The  Recurrence  Classification  of  Risk  and  Storage  Processes," 

J.  Michael  Harrison  and  Sidney  I.  Resnick,  September  1976. 

27.  "The  Generalized  Variance  of  a Stationary  Autoregressive  Process," 

T.  W.  Anderson  and  Raul  P. Mentz,  October  1976. 

28.  "Estimation  of  the  Parameters  of  Finite  Location  and  Scale  Mixtures," 

Javad  Behboodian,  October  1976. 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (When  Dole  Sntormd) 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 
|2.  GOVT  ACCESSION  NO.j  3-  RECIPIENT’S  CATALOG  NUMBER 


4.  TITLE  (and  Subtitle ) 

ESTIMATION  OF  THE  PARAMETERS  OF  FINITE 
LOCATION  AND  SCALE  MIXTURES 


7.  AUTHORfsJ 

Javad  Behboodian 


5.  TYPE  OF  REPORT  4 PERIOD  COVERED 

Technical  Report 

S.  PERFORMING  ORG.  T?EPORT]7uMiiR~ 
's.  CONTRACT  OR  GRANT  NUMBER!.]  ‘ 

N00014-75-C-0442 


9-  PERFORMING  ORGANIZATION  NAMeTnD  ADDRESS 

Department  of  Statistics 
Stanford  University 
Stanford,  California  9U305 

”•  CONTROLLING  OFFICE  NAME  AND  ADDRESS  ~ ’ 

Office  of  Naval  Research 

Statistics  & Probability  Program  Code  k36 

Arlington , Virginia  22217 

",4‘  MONITORING  AGENCY  NAME  a ADDRESS^!/  dllletent  from  Controlling  Office) 


16.  DISTRIBUTION  STATEMENT  (o'i  this  Report) 


10.  PROGRAM  ELEMENT,  PROJECT  TASK 
AREA  & WORK  UNIT  NUMBERS  ' 


(NR-0U2-034) 


12.  REPORT  DATE 

October  1976 

13.  NUMBER  OF  PAGES  ’ 

11 

15.  SECURITY  CLASS,  (oi  this  report) 

Unclassified 

IS«.  OECLASSI  F|  CATION/1  DOWNGRADING 
SCHEDULE 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


17.  DISTRIBUTION  STATEMENT  (a I the  ebstracl  entered  In  Block  30,  U dltUrent  Iron,  Report) 


18.  SUPPLEMENTARY  NOTES 


9-  KEY  WORDS  (Continue  on  reverse  aide  il  necoaamry  and  Identify  by  block  number)  ~ * 

Location  and  scale  mixtures,  ident if i ability , estimation  of  parameters. 


20.  ABSTRACT  (Continue  on  reverse  tide  II  neceeeary  and  identify  by  block  number)  ' 

A finite  location  mixture  is  represented  as  the  sum  of  two  independent 
random  variables,  and  a finite  scale  mixture  is  represented  as  the  product 
of  two  independent  random  variables.  Using  these  representations,  it  is 
shown  how  to  generate  random  samples  from  these  mixtures , how  to  prove 
their  ident if iabilities , and  how  to  estimate  the  unknown  parameters. 


1 JAN  73  1473  EDITION  OF  1 NOV  65  1$  OBSOLETE 

S/N  0 102-014-  6601  I 


Unclas s i f i ed 

SECURITY  CLASSIFICATION  OF  THIS  PAGE  f»h»n  Del*  Entered) 


